Reverse mapping method and apparatus for form filling

ABSTRACT

In the presently preferred embodiment of the invention, every time a user submits a form the client software tries to match the submitted information with the stored profile of that user. If a match is discovered, the program tags the field of the recognized data with a corresponding type. The resulting profile can be used after that to help all subsequent users to fill the same form.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.10/631,955 filed 30 Jul. 2003 now U.S. Pat. No. 7,779,345 which isincorporated herein in its entirety.

BACKGROUND OF THE INVENTION

1. Technical Field

The invention relates to completing a Web-based form with informationfrom a user's profile. More particularly, the invention relates to areverse mapping method and apparatus for form filling.

2. Description of the Prior Art

Form-fill is a technology that is aimed to automate the task of Web formcompletion with a user's specific profile information. Almost all Websites today collect some information from their users. This informationcould be addresses, names, emails, or even credit card information incase of on-line shopping. For a particular user in most of the casesthis involves the user entering the same address or credit cardinformation over and over again. This tedious and error prone processfor even Internet savvy users could become prohibitively difficult fornovices.

Enter the form-filling technology. It is usually implemented on theuser's client, e.g. browser, as a piece of software that knows theuser's profile data, such as his first name, last name, street address,phone, etc. It also has a mapping between the user's information and thecorresponding fields of the Web form to be filled. The form-filling codethen takes information from the user's profile and automaticallycompletes the form with the requested information. The user after thatcould review, i.e. proof read, the form and simply submit it.

The users' profile data is provided by the user upon the initial setupprocess or sometimes it can be collected on demand, based on therequirements of the current Web form. The most sophisticated part of theprocess is in creation of the mapping between the form fields and theirmeaning. This mapping process is presently known to comprise either of ahard-coded mapping and an intelligent mapping. Hard coded mapping isachieved by manually creating links between the field names and theirmeanings. Intelligent mapping uses a rule-based approach to search forkeywords that would help the software to translate the labels and thetext around the field.

Hard-Coded Mapping (Site Profiling)

A major issue with this approach is the cost to collect and maintainprofile information for an enormously large set of domains. The cost tosupport even the top 1000 domains could become prohibitively high. It isthought that this approach is impractical to maintain a list of morethan 300 supported domains.

Intelligent Form Fill (IFF)

The problem with IFF is that this can be characterized as naturallanguage understanding, and in some cases even image analysis, both ofwhich are known to present extremely difficult technical issues. Effortsover the last several decades in trying to recreate these simple humanabilities with software tools have not been particularly successful.Although, the problem to be solved is limited to recognizing of 10-15profile attributes correctly, the number of different ways in which Websites can ask for the same information is still very high. Algorithmsknown today rely upon dictionaries of keywords and patterns to identifythe form fields. Yet, the accuracy of such algorithms is just a littleabove 85%. Every additional percent in accuracy improvement requiressignificant trial-and-error, fine-tuning, and regression testing.Additionally, localizing these dictionaries and word patterns in otherlanguages would require the same time-consuming trial-and-errortechnique. This approach is thus considered to be practicallyinefficient in achieving and maintaining levels higher than 86-87%accuracy. It is also hard and expensive to extend the service tointernational languages.

It would be advantageous to provide low-cost, high quality form filling,with coverage of a large number of Web sites, and thereby overcome thelimitations of the existing site profiling and IFF solutions.

SUMMARY OF THE INVENTION

The invention provides a method and apparatus for low-cost, high qualityform filling, with coverage of a large number of Web sites, and therebyovercomes the limitations of the existing site profiling and IFFsolutions. In the presently preferred embodiment, every time a usersubmits a form the client software tries to match the submittedinformation with the stored profile of that user. If a match isdiscovered, the program tags the field of the recognized data with acorresponding type of the data submitted by the user. The resultingprofile can be used after that to help all subsequent users to fill thesame form.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a tabular representation of a reverse mapping of user datacontained in a submitted form to a result mapping according to theinvention;

FIG. 2 is a block schematic diagram that shows a high level architectureof the presently preferred embodiment of the invention;

FIG. 3 is a flow diagram that shows an auto profiling algorithmaccording to the invention; and

FIG. 4 is a flow diagram that shows an auto-correcting algorithmaccording to the invention.

DETAILED DESCRIPTION OF THE INVENTION

For purposes of the discussion herein, the terms in Table 1 below shallhave their associated meanings:

TABLE 1 Glossary of Terms Form (Site) Manual process of buildingrelations between the mapping fields in a given form and theircorresponding types (meanings). Form fill Technology that automates theprocess of Web form completion with user specific information, such asaddresses, credit cards, etc. Intelligent Form fill, which uses anheuristic, rule-based Form Fill approach relying on key words, patterns,and text (InteliFill) analysis to match the Web form fields with theirmeanings. This does not need Site Mapping, but its accuracy istheoretically less then 100%. Level-one Subscription networkauthentication with the authentication regular account screen name andpassword, which is used to gain network access to the subscriptionsystem and services. Level-S Wallet authentication advancing beyondLevel-one authentication authentication by providing either a walletpassword or answering a question/answer. One-way A one-way hashfunction, also known as a hash message digest, fingerprint, orcompression function, is a mathematical function which takes avariable-length input string and converts it into a fixed-length binarysequence. Furthermore, a one- way hash function is designed in such away that it is hard to reverse the process, that is, to find a stringthat hashes to a given value (hence the name one-way.) A good hashfunction also makes it hard to find two strings that would produce thesame hash valueIntroduction

Subscription services, such as AOL of Dulles, Va., have large userbases. Every day thousands of subscription service users fill in forms,using their shipping and billing information, while they register orshop on-line. During this process they use their human intelligence, toread, understand, and complete such forms. One aspect of the inventionborrows and reuses their intelligence to profile the pages they arevisiting. Thus, in the presently preferred embodiment, every time a usersubmits a form (see FIG. 1) the client software tries to match thesubmitted information 12 with the stored profile of that user 10. If amatch is discovered the program tags the field of the recognized datawith a corresponding type. The resulting profile 14 can be used afterthat to help all subsequent users to fill the same form. This approachis referred to herein as reverse mapping (RM).

The herein disclosed method and apparatus for reverse mapping of formsby borrowing intelligence from a large subscription service user base issignificant in at least the following ways:

-   -   It lowers the cost of maintaining high quality service across        large number of constantly evolving Web sites.    -   It is self-correcting, so that only a limited number of users        suffer degradation of their experience. As these users are        exposed to potentially erroneous mappings their corrective        measures are captured and reused subsequently to improve the        service for the rest of the subscription service population.    -   It is self-organizing because the most frequently visited sites        are mapped very fast and migrated from an unsupported category        to a supported category.    -   It works automatically for international Web sites using the        international users' profile information seamlessly.    -   It can be extended to arbitrary set of user specific profile        attributes.    -   It has a network effect because it becomes more accurate and        wide range as there are more users of the service.        Architecture

FIG. 2 is a block schematic diagram that shows a high level view of thearchitecture 20 of a presently preferred embodiment of the invention. InFIG. 2, the blocks Form Fill Engine 23 and Reverse Mapping engine 25both constitute a quick completion form fill (QCFF) client engine 22,which is part of the browser in the preferred embodiment. Those skilledin the art will appreciate that such components of the invention can besituated elsewhere within the overall system architecture, as desired.The presently preferred Form Filling Engine is implemented using acombination of C++ and JavaScript libraries, but other techniques may beused, as are known to those skilled in the art.

The Form Fill Engine (FFE) 23 is responsible for two major tasks, i.e.analyzing the context of the users' navigation, and completing the formsupon users' consent or user initiated actions. To complete these tasksthe engine pulls the user data and the rules to map this data to therequested form.

The FFE connects to the Site Profile Server (SPS) 24 to get information11 about the mapping of the site currently visited by the user from thesite profile database 30. It also connects to the User Data Server 26 toget user's profile 13 from the user database 32.

The Reverse Mapping Engine (RME) 25 is responsible for implementing thetwo reverse mapping algorithms described below in connection with FIGS.3 and 4.

After the engine recognizes a match, the extracted knowledge from thismatch 15 is communicated to the Reverse Mapping Server (RMS) 28 forpermanent storage into the Reverse Mapping Database (RMDB) 34.

Finally, the Consensus Engine 36 is a process running in parallel andconstantly monitoring the statistics of the reverse mappings, which arecollected. After passing certain statistical level of consensus, theConsensus Engine can decide whether the mapping is accurate. Thistriggers migration of the mapping from the Reverse Mapping Database 34to the Site Profile Database 30. The mapping could comprise variousforms of information, such as a completely new profile, or an update ofan existing profile. Following a successful migration/update, the enginepreferably resets the statistics in the Reverse Mapping Database 34 forthis form.

Exemplary Mathematical Foundation for Consensus Determination in ReverseMapping

Problem Description

The problem can be formulated as follows:

A set of classes associated with Web form fields and user profile datais defined as C={c₁, c₂, . . . , c_(n)}. A field can belong to a singleclass only. In the context of reverse mapping the classes can be FirstName, Last Name, Zip Code, Credit Card Number, etc. Let F be a givenform defined with its fields as F={f₁, f₂, . . . , f_(k)}. Where f_(i)corresponds to the name of the field. For practical reasons it is knownthat K, the number of fields in F is smaller then the number ofavailable classes. This is a correct assumption as some of the classesare potentially redundant, when combined with others. For example, theclass for the complete phone number (3+7) is redundant with the class ofthe area code (3) combined with the class of the local phone number (7).It is unlikely to encounter a Web form, which asks for the full phonenumber and separately for the area code and the local number. For thepurposes of this discussion let N—the number of classes be 50 and K—thenumber of form fields be 8. It is also known that the classes of thefields are unique, so there is at most one class of each. Let D be a setof complete user data defined as D={d₁, d₂, . . . d_(n)}. There is oneto one relation between the individual data members of D and the classesC, which means class(d_(i))=c_(i).

Define a reduction function R={r₁, r₂, . . . , r_(n)} over the data ofthe user profile. The purpose of the reduction function is to hash thedata and prevent attempts for identity or credit card theft. The hashingshould lead to losing of more than 90% of the useful information in theprofile, making the reconstruction of the original data practicallyimpossible. The hashed or reduced user profile is then produced byapplying the reduction function over the profile H=R(D)={r(d₁), r(d₂), .. . , r(d_(n))}. A simple example of R is an MD5 hashing followed by amod M operation. M could be class specific or more precisely class spacespecific. For example, for classes which occupy a relatively smallspace, e.g. month or year of the credit card expiration date, pick asmall M to guarantee that enough information is lost during thereduction. Call M the level of discrimination. The higher the level ofdiscrimination, the more information is preserved after the hash isapplied. For the purposes of the discussion let M be 4.

In addition, there is a large pool of unique users. Each of the usershas on average L number of D data profiles, where one of the profiles'reductions is known to the system. For the purposes of this discussion,let L be 3. In practice this means that the average consumer would usethree different credit cards and would ship to three differentaddresses, but only one of the addresses and one of the credit cardswould be known to the system. Every time a user from the pool submitsthe form F, capture the submitted data V={v₁, v₂, . . . , v_(k)} andapply R over it as G=R(V). Then compare the resulting reduced data Ewith the one known about this user and build match matrix M=H×V based onthe following rule:m_(ij)=0 if v_(i)!=h_(j) orm_(ij)=1 if v_(i)=h_(j).  (1)

Finally define the questions:

-   -   Is it possible to achieve arbitrary high accuracy in identifying        the classes of the form fields based on the matching results        received from the pool of users?    -   If it is necessary to ensure, e.g. 99.5% accuracy, how many        samples is it necessary to collect?        Mathematical Background—Bayesian Theory of Sequential Hypothesis        Testing

The following analysis relies on the Bayesian theory applied forhypothesis testing. The hypotheses herein that a given field f_(p) fromthe form F is of class c_(q). Evidence to prove the hypothesis is theresulting match matrixes after observing and collecting data from thepool of users. The classic form of Bayesian theorem is given as:

$\begin{matrix}{{P( { A \middle| E ,b} )} = {{P( A \middle| b )}\frac{P( { E \middle| A ,b} )}{P( E \middle| b )}}} & (2)\end{matrix}$where

-   -   A is the hypothesis under test;    -   b is context information representing knowledge about the system        before any evidence collecting;    -   E is a new evidence, that might change knowledge about the        system;    -   P is probability; and    -   | is used for notation to describe conditional probability.

With the above notations the Bayesian theorem can be interpreted as: Howknowledge/belief about the system has changed after new evidence E ispresented. P(A|b) is the old knowledge about the probability that A istrue, before the evidence E to be presented. P(E⊕A,b) is the probabilitythat E is true assuming the hypothesis A is true. And, finally, P(E|b)is a normalizing factor given as probability of E to occur independentlyfrom A. The less likely is E, the higher the weight of the evidence is,and vice versa.

Write equation (2) above with respect to finding how the evidence Echanges a belief about the probability that A is false:

$\begin{matrix}{{P( { \overset{\_}{A} \middle| E ,b} )} = {{P( \overset{\_}{A} \middle| b )}\frac{P( { E \middle| \overset{\_}{A} ,b} )}{P( E \middle| b )}}} & (3)\end{matrix}$

Now divide mutually (2) and (3):

$\begin{matrix}{\frac{P( { A \middle| E ,b} )}{P( { \overset{\_}{A} \middle| E ,b} )} = \frac{{P( A \middle| b )}{P( { E \middle| A ,b} )}}{{P( \overset{\_}{A} \middle| b )}{P( { E \middle| \overset{\_}{A} ,b} )}}} & (4)\end{matrix}$

The left side of (4) is called the “odds” of hypothesis A to be true incase the evidence E is presented. Similarly, P(A|b)P(E|A,b) can becalled the prior “odds” about A. Use notation O(A|E,b) for posterior andO(A|b) for prior odds about proposition A. Then rewrite (4) as:

$\begin{matrix}{{O( { A \middle| E ,b} )} = {{O( A \middle| b )}\frac{P( { E \middle| A ,b} )}{P( { E \middle| \overset{\_}{A} ,b} )}}} & (5)\end{matrix}$

In other words, the ratio

$\frac{P( { E \middle| A ,b} )}{P( { E \middle| \overset{\_}{A} ,b} )}$gives how presenting evidence E changes the odds of A. Note that

$\frac{P( { E \middle| A ,b} )}{P( { E \middle| \overset{\_}{A} ,b} )}$can be greater than one, in which case the evidence supports theproposition or less than one, in which case the evidence contradicts theproposition.

In the reverse mapping context, there is a sequence of evidence. The waythis is organized guarantees that the evidence E₁, E₂, . . . , E_(s) areindependent from each other. In this case, the cumulative effect of theseries is given as:

$\begin{matrix}{{O( { A \middle| E_{s} ,b} )} = {{O( A \middle| b )}{\prod\limits_{{i = 0},S}\frac{P( { E_{i} \middle| A ,b} )}{P( { E_{i} \middle| \overset{\_}{A} ,b} )}}}} & (6)\end{matrix}$

Now transform (6) to its logarithmic form as:

$\begin{matrix}{{10\log_{10}{O( { A \middle| E_{s} ,b} )}} = {{10\log_{10}{O( A \middle| b )}} + {10\;{\sum\limits_{{i = 0},S}{\log_{10}\frac{P( { E_{i} \middle| A ,b} )}{P( { E_{i} \middle| \overset{\_}{A} ,b} )}}}}}} & ( {6a} )\end{matrix}$

Call the new logarithmic representation evidence. There are well-definedtransformation rules from probability to odds to evidence as given inTable 1.

Evidence e (db) Odds O Probability P 0 1:1 1/2 3 1:2 2/3 6 4:1 4/5 1010:1  10/11 20 100:1  0.99  30 1000:1   0.999Simple Case—Single Profile Usage

To simplify the initial analysis of the problem, lower the complexity byassuming that the users stick to use data only from the profile, whichis known to the system. The task then is formulated as follows.

Problem 1:

The Web form F is defined with its eight fields f_(i), where each one ofthem corresponds to a class C_(j) from a set of 50 classes. There are notwo fields from the same class. There is a large pool of users' profilesD₁, D₂, . . . , D_(t). Apply the hash over the set of D as explainedabove with level of discrimination M=4. Then ask the users to completeand submit the form using their profiles. During the process ofsubmission the data are captured and hashed using the same function aswith the profiles. For every user, construct the match matrix M asdescribed in equation (1).

How many users are sufficient to achieve 99.5% accuracy of form fields'classification?

Solution:

The accuracy of 99.5% is for the whole form, which consists of 8 fields.To achieve total form accuracy of 99.5%, the accuracy per field shouldbe

$\sqrt[8]{0.995},$which is 0.9994. This translates to roughly 32 db per field in theevidence scale. The initial knowledge of the system is that there are 50classes. In the lack of any additional information, it is assumed thatthere is uniform probability of any of the classes. This sets the priorevidence

${e( A \middle| b )} = {{10 \cdot \log_{10}}\frac{1}{50 - 1}}$or −17 db. Each of the captured samples provides equal additive evidencewhich value can be expressed as:

$\begin{matrix}{{10 \cdot \log_{10}}\frac{P( E \middle| {Ab} )}{P( E \middle| {\overset{\_}{A}b} )}} & (7)\end{matrix}$

The way the event E is defined in the case of reverse mapping is by thematch between the user profile data hash and the submitted data hash. Ifthe classes of the profile data and submitted data are the same, andassuming the user uses only single profile, then the probability of thisis exactly one. In the denominator, the class of the submitted data andthe class of the hypothesis testing data are different. Because the userdata is hashed with information reduction, the space of possible Mvalues is much smaller than the space of the original values. This meansthat hundreds of thousands of different names, addresses, credit cardnumbers, etc are represented with the same code. In case of M=4 theseare 0, 1, 2, and 3. This gives a probability of

$\frac{1}{M}$false positive that the hashed first name of the user is classified asany one of the other 49 classes. Substituting the numbers in (7) gives aresult that the contribution of each new sample is 6 db. To achieve theneeded 32 db evidence, capture somewhat more than eight samples, whichis the answer of the problem.Multiple Profiles Usage

The numbers from the discussion above are contingent on the assumptionthat the users always select the profile data which is known the system.This is almost true when the data are related to the users' addressesbecause for the majority of on-line transactions these are the users'home addresses. However, in many cases an office address or somerelatives' or friends' addresses are used. This is even more doubtfulwhen the data are related to a credit card. The average U.S. adultconsumer has in his pocket on average five credit cards. Taking thisfact into account leads to the following questions: How the lack ofknowledge about the profile in use changes the convergence ratecalculated above?

This redefines the problem to a new one.

Problem 2

All the conditions from Problem 1 are still valid, but there is oneadditional factor that from the profiles D₁-D_(r) only one out of fiveis known to the system.

Solution:

The new condition does not change the prior evidence calculated above as−17 db. What is the evidence in case the user submits a known profile.Again, this does not change the value calculated previously as 6 db ofevidence contributed by each known profile. Estimate the evidence incase the profile is unknown. This time the value of the nominatorP(E|A,b) is the same as the value of the value of the denominatorP(E|Āb)=1/M. The explanation of this is as in the case of no knowledgeabout the data that are submitted, where there is 1/M possibility thehash of this is the same as the hash of the data used in the comparison.This gives as evidence value 0 because there is no correlation betweenwhat the system knows and what is currently submitted. It is stillnecessary to accumulate about 50 db of evidence on top of the −17 db ofprior evidence. However, this time the number of samples is much higherbecause only one out of five contributes to the total knowledge. Theother 80% of the captured data creates white noise. There is 50 dbdifference needed between the winner class and the highest peak of thenoise. For cases where the number of samples is higher than 30, assumenormal distribution. The mean is given as μ=Np, where p is thepossibility for a hit and N is the number of data captures. Taking M=4from above gives a possibility p=0.25. The standard deviation isσ=√{square root over (Np(1−p))}. For all practical purposes, considerthat the white noise amplitude is in the range of μ±3σ. Taking intoaccount the 50 db (eight units) deference needed between the winnerclass and any possible random peak leads to a value of μ+3σ+8. Pick asample size in which at least 3σ+8 of the profiles are known to thesystem. For N=250, can calculate or σ=√{square root over(250.025.0.75)}=6.8. Based on that at least 28 known profiles areneeded. Using similar approach, estimate that for all practical purposesfor a size 250 the number of known profiles is μ±3σ=50±18.97. The lowerbound is bigger than 28, hence in this case size of 250 capturesproduces a correct reverse mapping.

Conclusions

Bayesian theory shows that by collecting multiple imprecisemeasurements, e.g. data captures in the case of reverse mapping, one canachieve arbitrary high degree of confidence about the meaning of theform fields. Two cases were analyzed in respect to the convergencenumber in order to reach very high confidence (more than 99%), e.g.simple where the users are limited to provide data from a single, knownto the system profile, and complex where the users are free to use fiveprofiles with only one of them known to the system. The simple case hasbeen shown to reach 99.5% confidence in nine data captures. In contrast,the complex case needs on average 100 data captures to reach comparablelevel. Average Internet users are somewhere in the middle between oneand five different credit cards usage in their on-line shopping. When itcomes to the shipping addresses, the number of different options is muchcloser to one, rather than five. Regardless of the above more optimisticassumptions, collecting 250 reverse mapping captures is quite feasibleand can be used as a triggering point. In addition to providing a highconfidence level, this shows that the Web form is in high usage anddeserves the resources for its migration to the mapped forms.

Reverse Mapping Algorithms

The following discussion describes the reverse auto-profiling algorithm(see, also, FIG. 3):

-   -   For every form the user submits, collect the field names and the        corresponding values that the user has entered into the fields        of the form (100).    -   Compare the values entered in to the form with the same user's        data found in a central subscription service database (110).    -   If a match between the submitted data and the user's profile is        found, consider this as a hit. After that, mark the name of the        hit field according to the type of the user's data that it        matches (120).    -   Repeat these steps until a statistically reliable confidence        level about the real meaning of the fields is reached (130).    -   At that moment, migrate the form to a mapped status (140).    -   All subsequent users visiting the same page after that receive        service based on the profile built by learning from the initial        set of users (150).

The following discussion describes the reverse auto-correcting algorithm(see, also, FIG. 4):

-   -   For every form automatically filled, observe whether the user        makes corrections after the form fill is completed (200).    -   Try to match the corrections with knowledge about the user data        (210).    -   If a match is found, it is considered a hit. Re-map the field        that was corrected (220).    -   After statistically significant number of corrections are        introduced, migrate the corrections to the profile of the site        (230).        Client Cache Algorithm

Note: The caching algorithm is not required to achieve the value ofreverse mapping. It is an improvement with generic usage, and can beused by much broader set of applications. The cache described belowimproves the performance of the application, but it can be usedindependently from reverse mapping. In fact, the suggested cache can beimplemented with known solutions and still have a beneficial effect onperformance.

The caching algorithm involves coordination between the client and theSite Profile Server (SPS). The client maintains a cache of the form fillmapping data it receives from the SPS. In addition, the client maintainsan index of all domains for which the SPS has form fill mapping data.The index is identified by a version number and contains a record foreach domain supported. The client periodically polls the server toidentify differences between the index version it has and the mostrecent index. The client uses this difference information to update itslocal index. When the difference information indicates that the mappingfor a given domain has changed, the form fill mapping data for thatdomain stored in the cache is removed, if it was present.

While the client is used for browsing the Web, it checks to see if ithas the mapping data for each domain it visits. If it navigates to adomain for which it has form fill mapping data, it operates under theassumption that the form fill mapping data that it has for this domainis the most current available, and it proceeds to use it to recognizecheckout forms and offer to fill them. If the client navigates to adomain for which it does not have form fill mapping data, it requeststhe form fill mapping data for that domain from the SPS, stores it inits cache when it is received, and proceeds.

This algorithm maintains a relatively up-to-date version of mapping datafor a given domain on those clients for which it is relevant, whileminimizing the frequency of retrieval of form fill mapping data. If theSPS stores mapping information for a checkout form on, for example,YOUHAVENEVERBEENHERE.COM but a given client never visits that domain,the form fill mapping information is never downloaded for that domain bythat client. More importantly, if the same client visits, for example,TOWERRECORDS.COM once per day, the client downloads the updated mappinginformation for this domain no more often than the mapping informationchanges for this domain or the polling period for cache index updates,whichever is less frequent.

Because form fill mapping data for a given domain changes relativelyinfrequently, and because most clients do not visit a vast number ofdomains, the largest cost for the SPS results from the cache indexupdate requests. A reasonable polling period might be three days,although the actual polling period is a matter of choice for thoseskilled in the art.

Security and Privacy

The reverse mapping technique herein disclosed works well if it isapplied to the majority of a subscription service user base. This meansthat a reverse mapping activity runs as a background process on everymachine trying to match the user data stored in the subscription servicesystem with data that the user submits while browsing. This raisesquestions about violations of privacy and security standards.

Security

The danger here comes from the need to download the user's profilewithout challenging the user with a higher level of authentication. Thismay give access to private and sensitive information just by knowinguser's screen name and level-one password. Most subscription serviceusers are not very careful with their credentials. However, in keepingwith the invention herein several measures can be taken to alleviatethis concern. In one embodiment, the profile is hashed before sending itto the client. The same hash is performed with the data that the usersubmits. In this way, equal strings before hashing are equal afterhashing, as well.

Unfortunately, this method does not provide adequate security for allpurposes because, in some cases, the space set of the possible user datais limited. For example, a hacker can try several thousands of zip codesand compare the produced string after hashing until finds out the realzip code that generates the same hash. However, it is possible tocombine the technique disclosed herein with some additional restrictionsto prevent this from happening.

One embodiment encrypts the user data with a user specific symmetrickey, stored in the User Database. Then, the encrypted profile is storedon the user's machine after the first time the user authenticates with alevel-s password from this machine. This is preferably not allowed ifthe user is logged in as a guest. This eliminates the chance that theuser's profile may be left on some publicly available PC, e.g. in alibrary or Internet cafe.

Another embodiment is useful if it is decided not to use all of theinformation available in the user's profile. For example, the system maytransform every single value form the user's profile to a couple ofbits, losing in this way most of the information. One possible way to dothis is to use a hashing method for the individual fields of the user'sprofile and then use only the last 2-3 bits of the hashed value. Thisprofile reduction generates a lot of false positives. However, a goodhash algorithm is known to produce uniform distribution of the hashedvalues, which in turn leads all false positives to negate each other. Incontrast, the true positives will consolidate around the same type ofthe field. This reveals only few bits of the information to hackers andstill provides enough confidence from a statistical point of view togenerate the reverse mapping.

Privacy

A major concern with use of the invention concerns privacy violationsthat could occur by monitoring the users and collecting informationabout their actions and habits without the users consent thereto. Thepreferred embodiment of the herein disclosed invention does not need toknow exactly who was performing certain form fills, but rather how, onaverage, a number of users understand the meaning of a given form. Inthis regard, the invention does not need to store any user identifiableinformation per se. On the other hand, there is a requirement to makedecisions based on the consensus built from a number of different userscompleting a given form. Also, it is desirable to prevent any chancesthat a single person can create a wrong, potentially malicious mappingby submitting incorrect data multiple times.

These seem to be conflicting concerns. One way to resolve this conflictis provided by the following approach. At the first time the userauthenticates with his level-s password, the server generates a large,random number token, signs it with a certificate, and stores itpersistently at the user's machine. The digital signature is used tovalidate the legitimacy of the ID. The user is then flagged to preventthe issuing of multiple ID's for the same user. The randomness of the IDguarantees that real user identities are not traceable. In this way, theRMS can identify communications for unique users without knowing theactual identity of the users themselves.

This ID and signature are then included in all communications betweenthe user and the Reverse Mapping Server (RMS). The number should belarge enough to identify the user uniquely. The use of a signedcertificate prevents tampering.

The signed token is stored at the user's machine along with the profile.This becomes the identification mechanism needed to ensure the consensusrequirement. Because the original number and the signed number are notstored in the server at the moment of their generation there is nopractical way to trace back the real user name behind this profile.

It is still theoretically possible that the same user can installmultiple browsers and perform multiple wrong mappings. To prevent this,the number of times a given user is assigned a new token ID can belimited. This number is optionally reset periodically so that users whonormally change their machines or upgrade are not dropped from thenetwork of contributors.

Note: Similarly to the cache, the identity decoupling approach describedabove is an improvement or enhancement to the reverse mapping invention.It is a part of a preferred reverse mapping, but not from a technologypoint of view. Its main value is to elevate privacy/security concerns.It is part of the preferred embodiment, but can be applied in a muchbroader scope in similar cases where the exact identity of the user isnot needed, but rather it is necessary that guarantee the different orunique users get access to certain privileges. It is the election votingparadigm where there is a right to vote, but nobody can trace what thevote is.

Protection of Wallet Data

For reverse mapping to be available as part of the client, theopportunity must be available to the client to compare data entered inHTML forms with the data in the user's wallet. Because it is a rareoccurrence when the user is level-s authenticated, the walletinformation must be available for comparison when the user is only levelone authenticated. This presents some security concerns because theuser's wallet information is highly sensitive information. Great caremust be taken to ensure that the information is not compromised. This isaccomplished in the preferred embodiment by using the followingprecautions:

The user's profile is stored persistently on the client after the firstlevel-one authentication.

The user's profile is partial. For example, every value in the profileis hashed using MD5 and after that only the last three bits are send tothe client. This way, even if the information becomes available tohackers it does not provide them with any substantial benefit. Thiscreates significant number of false positives with the reverse mappingalgorithm. This effect can be filtered out by requiring a minimum numberof users, 100 for example, to produce the same mapping. Even if thisgenerates a lot of false positives, they tend to be uniformlydistributed. Only one of the mappings stand up higher than the others,and this is the one around which the users consolidate.

False Mapping Vulnerability

Because the RM process relies on the user's activity to deduce the formfields' types there is a possibility for someone to create incorrectmappings to gain unfair benefit. For example, a malicious user canprofile his own site, where there is a form for visitors to register.The visible part of the form would ask for some harmless user data, suchas zip code or state, but the hidden part would require credit card andbilling address information. There are a number of ways to accomplishthis, for instance the hidden fields could be off-screen in a locationthe user would not expect to look, or they could be concealed using theCSS visibility or z-order styles. The user might allow form-fill to fillthe form with their sensitive information and submit it, withoutrealizing that this was happening.

This risk can be alleviated if the server side of the RM waits until aconsiderable number of unique users generate the same mapping for theform. Only after a certain threshold number is reached would the serverenable the new mapping for all users. In this way, a hacker alone wouldnot be able to pass the threshold, but would need a large number ofaccomplices to help him.

Additional protection could be provided if every form-fill completion isaccompanied with a message informing the user what information is aboutto be filled. Requiring the user's consent before the actual form-fillhappens, including options for the user to check/uncheck which data areto be filled, could further strengthen this method.

Exact Match of Wallet Items Vs. Inexact Match

This embodiment of the invention comprises an algorithm for storing onlythe obscured wallet data items and not the actual wallet itemsthemselves. This algorithm can recognize wallet data items because, whenthe same transformation is applied to the values entered in HTML formsby a user, the result is always the same for the same input value. Thus,if a user's wallet contains the first name “George,” then every time theuser types the name “George” into an HTML form, the field containingthis exact value is correctly identified as being the first name type.

Take, for example, the case when the user types his first name as“GEORGE.” This is not an exact match, and thus is not correctlyidentified as a first name field. An even more complicated example isthat of the phone number field. If the phone number is stored in thewallet as “555-1212” but the user enters “5551212,” again the data donot match exactly, and the comparison fails. One solution to thisproblem is to normalize the data in both cases before performing thetransformation. The wallet data and the data entered in the form couldbe first converted to upper case and then spaces and punctuation couldbe removed before the transformation is applied. The result is then thesame, and the comparison succeeds despite differing formats. The formatused could be communicated to the RMS along with the field type so thatit can later be used by the Form Fill Engine (FFE) to fill the form byformatting the data in the way the form expects.

Multiple Selection Input Items

Even when using some kind of normalization process, it may still not bepossible always to make an accurate comparison. Multiple selection formitems are good examples of this. For instance, if the wallet stores thecredit card type as “Visa” but the credit card type in a form is to beentered in a drop down list having the options “Visa Account,”“MasterCard Account,” and “Amex Account,” no reasonable normalizationprocess could provide an exact match for the wallet item and the formitem.

The best coverage for this situation is provided when combining the IFFand RM solutions. IFF can be used to make guesses about the presence ofcheckout forms and the field meanings when the mapping for a form isunavailable from the SPS. The information about field type from IFF canthen be included in the reverse mapping information sent to the RMS. Asan additional benefit of this approach, the user does not have to waituntil other users have reverse mapped a given form before form-fill isavailable.

Although the invention is described herein with reference to thepreferred embodiment, one skilled in the art will readily appreciatethat other applications may be substituted for those set forth hereinwithout departing from the spirit and scope of the present invention.Accordingly, the invention should only be limited by the Claims includedbelow.

The invention claimed is:
 1. A method comprising the steps of: a clientmaintaining a cache of form fill mapping data that it receives from asite profile server, the form fill mapping data created using reverseauto-profiling based on form submissions by a plurality of users; saidclient maintaining an index that contains a record for each domain forwhich said site profile server has form fill mapping data, wherein saidindex includes one or more index version identifiers; said clientperiodically polling said site profile server to identify one or moredifferences between the one or more index version identifiers at saidclient and one or more most recent index version identifiers at saidsite profile server; said client using said one or more differences toupdate said client's local index; and removing form fill mapping datafor a domain stored in said cache, if it was present, when saiddifferences indicate that a mapping for said domain has changed.
 2. Themethod of claim 1, further comprising the steps of: while said client isused for browsing, said client checking to see if it has cached formfill mapping data for each domain it visits; if said client navigates toa domain for which it has cached form fill mapping data, said clientoperating under an assumption that form fill mapping data that it hasfor said domain is the most current available, and said clientproceeding to use said form fill mapping data to recognize one or moreforms of the domain and to offer to fill one or more fields of the oneor more forms; and if said client navigates to a domain for which itdoes not have cached form fill mapping data, said client requesting formfill mapping data for said domain from said site profile server, andsaid client storing said form fill mapping data in said cache when saiddata are received.
 3. The method of claim 1, wherein the form fillmapping data comprises one or more mapping data entries for one or morecorresponding domains, each mapping data entry including one or moremappings between one or more form field names for the correspondingdomain and one or more corresponding data classes.
 4. The method ofclaim 1, wherein the reverse auto-profiling based on form submissions bythe plurality of users comprises a consensus determination for theplurality of users based on a Bayesian analysis.
 5. The method of claim1, wherein the reverse auto-profiling based on form submissions by theplurality of users comprises reverse auto-correcting based on the formsubmissions.
 6. A client apparatus comprising: one or more processors; aclient-based cache of form fill mapping data that said client apparatusreceives from a site profile server, the form fill mapping data createdusing reverse auto-profiling based on form submissions by a plurality ofusers; an index maintained at said client apparatus of a record for eachdomain for which said site profile server has form fill mapping data,wherein said index includes one or more index version identifiers; amechanism at said client apparatus for periodically polling said siteprofile server to identify one or more differences between the one ormore index version identifiers at said client apparatus and one or moremost recent index version identifiers at said site profile server; amodule at said client apparatus for using said one or more differencesto update said client apparatus's local index; and a module for removingform fill mapping data for a domain stored in said cache, if it waspresent, when said differences indicate that a mapping for said domainhas changed.
 7. The apparatus of claim 6, wherein: while said clientapparatus is used for browsing, said client apparatus is configured tocheck to see if it has cached form fill mapping data for each domain itvisits; if said client apparatus navigates to a domain for which it hascached form fill mapping data, said client apparatus is configured tooperate under an assumption that form fill mapping data that it has forsaid domain is the most current available, and said client apparatus isconfigured to proceed to use said form fill mapping data to recognizeone or more forms of the domain and to offer to fill one or more fieldsof the one or more forms; and if said client apparatus navigates to adomain for which it does not have cached form fill mapping data, saidclient apparatus is configured to request form fill mapping data forsaid domain from said site profile server, and said client is configuredto store said form fill mapping data in said cache when said data arereceived.
 8. The apparatus of claim 6, wherein the form fill mappingdata comprises one or more mapping data entries for one or morecorresponding domains, each mapping data entry including one or moremappings between one or more form field names for the correspondingdomain and one or more corresponding data classes.
 9. The apparatus ofclaim 6, wherein reverse auto-profiling based on form submissions by theplurality of users comprises a consensus determination for the pluralityof users based on a Bayesian analysis.
 10. The apparatus of claim 6,wherein the reverse auto-profiling based on form submissions by theplurality of users comprises reverse auto-correcting based on the formsubmissions.
 11. A non-transitory computer-readable media comprisingcomputer-executable instructions that, when executed by one or moreprocessors of a computer system, cause the computer system to: maintaina cache of form fill mapping data that the computer system receives froma site profile server, the form fill mapping data created using reverseauto-profiling based on form submissions by a plurality of users;maintain a local index that contains a record for each domain for whichsaid site profile server has form fill mapping data, wherein said localindex includes one or more index version identifiers; periodically pollsaid site profile server to identify one or more differences between theone or more index version identifiers at the computer system and one ormore most recent index version identifiers at said site profile server;use said one or more differences to update said local index; and removeform fill mapping data for a domain stored in said cache, if it waspresent, when said differences indicate that a mapping for said domainhas changed.
 12. The non-transitory computer-readable media of claim 11,further comprising computer-executable instructions that, when executedby the one or more processors of said computer system, cause thecomputer system to: while said computer system is used for browsing,check to see if it has cached form fill mapping data for each domain itvisits; if said computer system navigates to a domain for which it hascached form fill mapping data, operate under an assumption that formfill mapping data that it has for said domain is the most currentavailable, proceed to use said form fill mapping data to recognize oneor more forms of the domain and to offer to fill one or more fields ofthe one or more forms; and if said computer system navigates to a domainfor which it does not have cached form fill mapping data, request formfill mapping data for said domain from said site profile server, andstore said form fill mapping data in said cache when said data arereceived.
 13. The non-transitory computer-readable media of claim 11,wherein the form fill mapping data comprises one or more mapping dataentries for one or more corresponding domains, each mapping data entryincluding one or more mappings between one or more form field names forthe corresponding domain and one or more corresponding data classes. 14.The non-transitory computer-readable media of claim 11, wherein thereverse auto-profiling based on form submissions by the plurality ofusers comprises a consensus determination for the plurality of usersbased on a Bayesian analysis.
 15. The non-transitory computer-readablemedia of claim 11, wherein the reverse auto-profiling based on formsubmissions by the plurality of users comprises reverse auto-correctingbased on the form submissions.